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I. Introduction 

Instructional development is a process with compelling internal logic and proven appli¬ 
cability to the management of academic systems. Yet it is widely ignored, or misunder¬ 
stood, or misapplied. Indeed the misunderstanding is so pervasive that it is worthwhile 
pointing out what instructional development is not as a preamble to saying what it is. 

It is not a movement to sponsor wider use of audio visual aids, or television, or 
computers, or independent study, or any other specific instructional methods. 

It is not a system aimed primarily at saving money. 

It is not a system for sponsoring innovations for their own sake. 

It is not a discipline or a management science requiring new or highly technical 
competencies. 

It is quite simply a systematic way of applying to the design of instructional materials 
and programs the care and standards of critical thought taken for granted in all the 
scholarly disciplines. This claim will be expanded by dealing with three questions: What 
is instructional development? Why is it likely to be useful? How can its value be realized? 
However, because it is a process the what and how are almost inseparable, and because 
the process concerns itself very much with a critical linking of the objectives of a program 
to the methods, none of these questions stands alone. 

It is perhaps presumptuous to include How in the title since this paper in no way sets 
out to be a guide to the design of instructional programs. The steps in the process have 
been written about in detail elsewhere. All that is attempted here is to deal with certain 
general and central problems not dealt with in some of the more technical papers on 
course development, and to offer the reader an outline of the essentia] logical steps and 
relationships. 

Although instructional development can only be defined by a detailed examination 
of the whole process a condensed initial definition must be attempted. As used in this 
paper instruction refers broadly to any facilitation of learning. Development refers to 
change over time with progression toward some more mature or more adapted condi¬ 
tion. Adapted may refer to greater achievement, or fitting in more realistically with cost 
constraints, or having a more lasting influence on students. The criteria for adaptation 
will be complex and will vary with the circumstances. 


Professor of Biology, Queen’s University. Currently on secondment as Director, Ontario Universities 
Program for Instructional Development. 



34 H. M. Good 


In practice instructional development involves conscious effort to proceed “systema¬ 
tically.” Systematically as used here implies two things. First, a logical progression from 
careful enunciating of objectives, to development of a plan for achieving these, to an 
evaluation of success, to reconsideration of the objectives, to an application of the results 
of the evaluation to improving the process. Second, a conscious effort to ask all the 
relevant questions, i.e., to see the whole system into which the program being developed 
fits. 

The need to consider the whole system in which a program operates arises from the 
possibility that important feedbacks may occur producing effects which are either unex¬ 
pectedly amplified or produce influences the very reverse of those intended. The empha¬ 
sis on asking all the questions, i.e., developing a holistic or systems-conscious approach, 
and watching carefully for side effects, is a vital part of the concept of systematic develop¬ 
ment. 

The whole process is admittedly easier to describe than it is to follow. However, al¬ 
though there are formidable difficulties, the system can be made rigorously logical. Most 
current procedures in our universities fail signally to give the attention to internal logic 
and consistency implied in the systematic approach to design of teaching materials and 
programs. Given the ostensible, indeed often ostentatious, devotion of the university to 
“critical thinking,” this deficiency represents an inconsistency which should give us real 
concern. 

Deficiencies in the process for planning and developing teaching materials and approach¬ 
es are not of course fully generalizable. This paper is written in the context of the Canadian 
university system where the opening statements are, by and large, justified. However, in 
many universities (most notably in Britain, Australia and the U.S.A.) Centres for Instruc¬ 
tional Development have been established. These vary but normally provide advisory 
services on course development, evaluation, or program design. A few sources of informa¬ 
tion about these centres are listed at the end of this paper. 

II. Why? — The Potential Contribution 

Instructional Development as defined herein can contribute to a university in a variety 
of ways. Indeed from its emphasis on a holistic and internally consistent approach it can 
contribute to all phases of the operation. But there are costs, since the process is not a 
simple one and there are risks that it will be followed in a partial way which may be 
useful but may equally leave the whole system at the mercy of one key illogicality. With 
a caveat about the dangers of partial application, it is relevant to look at some areas in 
which there is evidence that the systematic approach holds out promise of a return which 
is considerably larger than the outlay. 

The areas considered fall into three general categories which are of course not entirely 
discrete. The first of these concerns the logic and consistency of institutional purposes 
and actions viewed in relation to the responsibilities of staff and their perceptions of 
these responsibilities, i.e., with accountability to Society. The second concerns Instruction¬ 
al Development and adaptability. The third deals with internal benefits stressing accoun¬ 
tability of staff and administration to other members of the University Community. 

Consistency of Institutional Purposes and Actions: There is a failure, widely recognized 
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by students and the public, that universities do not now achieve logical consistency in 
setting and pursuing broad policy. A look at certain aspects of staffing policy and rewards 
systems will illustrate this and point generally toward the kinds of action needed to cor¬ 
rect any discrepancies which might be diagnosed through the kind of analysis proposed. 

A university teacher may be said to have a responsibility to contribute to each of 
three interconnected sets of institutional objectives which we may designate as the In¬ 
stitutional-Societal, the Scholarship-Discipline, and the Student-Teaching. These may be 
represented by the three apices of a triangle. 


SCHOLARSHIP- 

DISCIPLINE 


INSTITUTIONAL- 

SOCIETAL 



STUDENT¬ 

TEACHING 


Any one staff member will weigh these responsibilities differently so that one might 
be represented by a dot in the middle of the triangle, another well toward the Scholar¬ 
ship-Discipline apex, and still another toward the Institutional-Societal apex. 

The main thrust of the past few decades, widely discussed in the literature but to 
some extent unrecognized by those who might have modified the trends, has been 
toward a heavy emphasis on the Scholarship-Discipline loyalties. In terms of our graphic 
representation the figure has become like this for most universities. 


SCHOLARSHIP- 

DISCIPLINE 


INSTITUTIONAL- 

SOCIETAL 



STUDENT¬ 

TEACHING 


36 H. M. Good 


There is nothing inherently wrong with this kind of distribution if it arises from a 
deliberate weighting of the three responsibilities for that university which is willing to 
justify this weighting to students and to society. However, the distribution of staff in 
this diagram is now, for many universities, appreciably out of balance with respect to 
either that pictured in statements of senior university officials or that considered accep¬ 
table to students or public. 

The argument that the Discipline-Scholarship loyalty feeds directly into the Teaching- 
Student loyalty making the distinction between these responsibilities unnecessary needs 
to be considered. However the frequent, and perfectly open, decisions by staff to econo¬ 
mize on teaching effort in order to provide research output because that is what will be 
noticed and rewarded makes it clear that the distinction is in practice a sharp one. This 
is not, of course, to say that active research interests do not have a valuable spin-off into 
teaching. It is simply to argue that effort going into research does not automatically lead 
to good teaching. Trying to rationalize teaching responsibilities on the basis of the argu¬ 
ment that it does is simply dishonest even if there are a few whose responsibilities are to 
teach only at a very advanced level and for whom the distinction does admittedly become 
blurred. 

Instructional Development as defined herein can be applied to consideration of 
staffing the overall system and can provide for a corrective feedback aimed at achieving 
an acceptable objectives — decisions match when the purposes of the institution and the 
distribution of staff in the triangle are visibly inconsistent. It should be stressed, however, 
that balance does not imply uniform distribution of dots but only distribution which 
is rational and effective in terms of the priorities of the institution being considered. 

This simple triangular schema can be used to sharpen questions about the weighting 
of institutional objectives, the criteria for selection of teaching or administrative staff, 
the priorities for different kinds of programs, and the design of appropriate evaluation 
criteria for rewards as well as for appointments. The distribution of staff in the triangle 
is clearly a function of both selection and rewards, and it can be altered significantly 
through adjustment of the rewards system. The schema is, of course, intended to be 
conceptual rather than a basis for a numerical formula. It is intended to present graphi¬ 
cally the kind of analysis of general policies through which Instructional Development 
or “systematic” analysis could make a contribution to the effective and efficient oper¬ 
ation of an institution. 

Instructional Development and Adaptability: The relation of instructional development 
to adaptation of the university to changing political or economic conditions, and espe¬ 
cially to the maintenance of the more important values when hard times impose specially 
rigorous financial constraints should be obvious; but from the common tendency to 
stand firm on traditional ground under these conditions it is clear that this connection 
has been generally missed. In times of affluence there is no pressure to be critical or 
experimental and in times of economic strain there is a tendency to take a defensive 
position behind the arguments — demonstrably false — that instructional development 
is a frill and impossible to apply without supplementary resources. 

Yet instructional development as defined in this paper is the sine qua non of effective 
adaptation and adjustment to minimal resources. Biological evolution provides a useful 
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analogy here. It leads to continued change and adaptation to the environment through 
mutations acted upon by rigorous selection. In instructional development critical ap¬ 
praisal of what we are doing leading to new proposals is the equivalent of genetic mu¬ 
tability and evaluation with corrective feedback is the equivalent of continuous natural 
selection. Leave either of these functions out of the adaptive system and it fails in logic 
and in effectiveness. 

The process of academic adaptation is therefore another way of defining instructional 
development. The need rises with the rapidity of change in the environment. In times 
of financial crisis it is more important, not less. The risks of getting one’s values out of 
order, or of pursuing policies for historic rather than rational reasons becomes greater 
under conditions of stress. The systematic approach to reappraisal of basic assumptions, 
to the ordering of values, and to the questioning of traditional methods then becomes 
of vital importance. 

In arguing the importance of systematic evaluation and reappraisal one is not, of 
course, arguing that what the universities now do will need wholesale revision. The logic 
of development of many specific disciplines is clear and demonstrably effective. But with 
respect to the deployment of teaching resources, the logic is less clear and indeed in some 
cases appears faulty. Since this is where the bulk of resources is expended, it is one area 
which invites prompt and critical reappraisal. It would, however, be entirely unsystematic 
to rush into the reconsideration of method without relating this critically to objectives 
and values just as it is unsystematic to hold so dogmatically and uncritically to values 
that methods cannot be looked at freely. Only out of an integrated reappraisal of values 
and methods can come improvement and adaptation. It is just such an approach which 
is the key to, and core of, instructional development. 

Instructional Development and Internal Accountability: To shift now toward the impli¬ 
cations of Instructional Development for the improvement of internal accountability, 
a further advantage of the systematic approach is the pinpointing of the real basis 
for decision making. Because systematic development requires that the rationale 
for each option be enunciated the basis for decisions can be challenged on logical 
grounds — often with the discovery that a decision to support project A has been re¬ 
jected while there are projects of lower priority being supported. The decision may, 
therefore, have been made by invoking a self-imposed or imaginary constraint perhaps 
to sidestep the need to eliminate programs which have been established but cannot be 
justified competitively. Admittedly, there are problems of responsibilities to personnel 
and of fair notice in discontinuing established programs but failure to do so puts the 
university in an evolutionary strait-jacket which can only lead to disaster. 

Another useful by-product of instructional development is improved effectiveness. 
Here it is appropriate to draw on an example in which the Ontario Universities Program 
for Instructional Development has been involved and to cite the cooperative develop¬ 
ment of the first year chemistry course by Professor David Humphries of McMaster 
University and Professor Ronald Martin of the University of Western Ontario. Achieve¬ 
ments of this project in 1973-74 were impressive. The failure rate of first year studies 
was reduced, average standing as judged by a comparable or identical exam was increased, 
and it was judged feasible to increase the content of the course significantly if this were 
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desired. There was also some evidence of development of better study habits. 

One more advantage is increased efficiency. Again the McMaster-Western chemistry 
project will serve as example. Quantitative data for claiming increased efficiency are less 
clear than for effectiveness but in the judgment of the departments involved the evidence 
is adequate. The increase in efficiency can be seen at University of Western Ontario in 
the substitution of one self-paced non-lecture course for nine parallel traditional lecture 
courses. The judgment of the department was that the parallel preparation and delivery 
of nine essentially identical lectures was not an efficient way to use teacher time. The 
self study material, while costly to develop, could, once developed, permit the redeploy¬ 
ment into interactive teaching or research, or whatever, of a significant part of the 
teacher time previously assigned to didactic presentation. 

Yet another advantage is improved student attitudes. Again this was demonstrated 
in the chemistry program and it has been shown in several redeveloped courses at Syra¬ 
cuse and at other universities. 

A final advantage is that some carefully redeveloped courses have shown considerably 
improved retention of learning. 

Indeed, successes have been fairly consistent when projects have been given adequate 
support and staff assigned who were competent managers and either familiar with the 
essentials of systematic development or knowledgeable enough to know when they 
needed help. Projects which did not meet these conditions have all too often failed. The 
moral is that the whole process works but fragments of it usually do not. This view is 
strongly supported by the experiences of Britain’s Open University, discussed in a later 
section. 

Summary of Advantages: The advantages claimed for systematic development range 
over a broad spectrum of the universities’ activities. Indeed they must do so if the whole 
system is kept in view. A corollary is that the benefits derived are not for any specific 
sector of the university but are distributed throughout and in a manner appropriate to 
the objectives of the institution. Thus, in the example cited above, students clearly pro¬ 
fited since they preferred the revised chemistry course and learned more. Moreover, they 
appreciated the fact that there was in the method of development a clear-cut way in 
which student comments and achievements or failures were cycled back into the revision 
process. This removes a major student concern that their efforts in evaluating courses 
are less effective than they might be because the information is not systematically used. 
Faculty profit because the pre-planned use of student reaction removes much of the 
threat of evaluation imposed by ad hoc student evaluations not linked to a formal and 
planned revision process. Faculty also profit by release of time from unproductive 
effort or from a type of instruction which they dislike. Finally, administrative officers 
profit because, as the rationale becomes clearer, waste can be reduced and brush fire 
or ad hoc administration replaced with more rational and more long range planning 
which is required to keep the university in a harmonious relationship to the society 
which supports it. 

III. How? — Steps in the Process and Some Central Problems 

If university programs are to be developed in a fully systematic way then there will have 
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to be an evolution of new working arrangements which, given the nature of the university, 
must be conceived as coming gradually. Some of these must develop in relation to the 
individual teacher, some at the departmental level, some at the institutional level. To 
attempt to discuss each level of the process lies outside the scope of this paper. Therefore, 
although differences in emphasis are recognized as appropriate to different levels, this 
section has been written without distinction between issues of primarily individual or 
departmental or institutional responsibility. The section covers: essential steps in the 
systematic method, the evaluation problem, and teaching — the choice of method (which 
includes discussions of the distinction between the textual and interactive systems). 

These are topics selected to illustrate the process more fully and to explain the basis for 
the conclusions and recommendations section. They do not attempt to provide compre¬ 
hensive coverage of the method. 

Steps in the Method: The essential steps of the method described herein as the essence 
of instructional development are: 

(1) Careful enunciation of objectives. (There is a temptation here to say precise 
instead of careful but this can lead to preoccupation with unimportant or even 
trivial objectives just because they can be stated and measured precisely). 

(2) Definition of the population to be instructed with special attention to assump¬ 
tions regarding prior knowledge. 

(3) Enunciation of the criteria for success in achieving the objectives. 

(4) Development of a system for determining whether the criteria for success have 
been met. This is essentially the development of the process for evaluation of both 
the instructional material and the students. In terms of instructional development, it 
is the evaluation of the materials rather than of the students which is the more impor¬ 
tant. This type of evaluation is almost inseparable from the evaluation of teaching 
and hence of teachers. This will be dealt with in more detail in the section dealing 
with the evaluation problem. 

(5) Examination of all ways of moving the population to be instructed from the 
level implied in the assumptions to the level implied by the statement of objectives 
and selection of the most effective overall mix of methods given the applicable con¬ 
straints. These steps imply the making of all the decisions involved in designing an 
instructional program to move the students from where they are to where it is hoped 
they will be. 

(6) Delivery of the instructional package to the students. 

(7) Application of the system of evaluation. 

(8) Iteration of steps 1 to 6, to eliminate unrealistic or trivial objectives, modify 
badly stated ones, or introduce new ones; and to change the methods of instruction 
or modify materials as needed. This is the corrective feedback loop of the systematic 
approach. It stresses the importance of evaluation of the instructional materials for 
modification and it implies a continuous rather than a one-shot process. 

Two ideas in these steps are of special importance. The first is the instruct-evaluate- 
modify cycle which is a central concept in instructional development. Indeed without 
careful attention to this cycle the word development has no meaning. 

The second concept which is central to Instructional Development is imbedded in 
step 4 which calls for an examination of all possible methods and selection of an optimal 
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mix. This requires the holistic or systems-conscious viewpoint referred to in the Intro¬ 
duction and implies a careful ordering of decision making so that the inevitable trade-offs 
can be made wisely. It involves a form of academic management which is today sadly 
lacking in universities. 

There is no implication in any of the steps outlined that new methods should be used 
although there is an implication that new as well as old methods will be looked at cri¬ 
tically to determine which are most appropriate. Much “innovation” in teaching is little 
better than isolated and ad hoc experimentation. Although such experimentation may 
on occasion score a spectacular success it lies outside instructional development as defined 
in this paper. 

In this brief account it is not possible to deal with each step of the development 
process fully. Much has been written about the formulation, use, and abuse, of objec¬ 
tives. A pious and formal enunciation of objectives cannot of itself do much. But with¬ 
out reasonably clear objectives there can be no effective collaboration between staff 
members, or between staff and students, and there can be no valid judgements about 
success either of students or teachers. Clarification of objectives is essential to effective 
communication. Important questions about whether duplication of effort is legitimate, 
about what is best eliminated if the pinch is really on, and a list of other decisions 
requiring critical discussion compel careful attention to objectives or the discussion 
becomes diffuse and useless. Participants are talking at, rather than to, each other. 

Regrettably, the enunciation of objectives has been developed by many as a way 
of formalizing the teaching and examining in a specific course and this formalization 
has often been carried to an extreme which stressed trivial matters. “Clarification of 
objectives” is a phrase which often produces a negative reaction. Yet the decision making 
process has no logical integrity if this is omitted or done sloppily. Stressing the major 
objectives or aims as an aid to effective communication between colleagues who are, at 
least in their own disciplines, trained to smoke out inconsistencies and intellectual inade¬ 
quacies, seems a useful and valid aspect to stress in developing this part of the system. 

Several of the problems inherent in steps 3, 4, and 5 need to be considered since they 
arise frequently and bear on the areas in which Instructional Development may be most 
valuable. They are important in themselves, and are appropriate starting points for a 
systematic review. It is true that, once started, the iterative systematic method expands 
to cover the system. However, the selection of points of entry is by no means inconse¬ 
quential. Hard thought is required and it is not likely to be provided unless the issues 
initially looked at are seen as of immediate interest and practical significance. 

The topics selected for illustrative discussion are Evaluation which bears directly on 
steps 3 and 4, and Textual Materials and Interactive Teaching which deal with the use 
of teacher time and bear on step 5 in the process. 

The Evaluation Problem and a Possible Model: The case for careful and critical evaluation 
is simply that without it we have no basis for distinguishing progression from retrogres¬ 
sion. Unless one is prepared to argue that university instruction is perfect both for the 
present and the foreseeable future and that change is to be actively discouraged, one 
must argue that some measure of change aimed at improvement is needed. Improvement 
implies not just one evaluation but two: of what we have been doing before and what 
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we are doing now. Only a comparison of these two evaluations can indicate whether 
things are better or worse. Change not aimed at improvement is mindless and frivolous. 
Claims of improvement without evaluations are empty. We have simply got to have 
evaluation or one cannot hope for improvement. Certainly we cannot operate a systema¬ 
tic approach to development. 

The above is likely to be widely misunderstood. It is likely to be interpreted as support 
for a movement, particularly strong in the United States, to develop and count heavily 
upon specialists in evaluation. But it will be clear from the model set out below that I 
believe that the delegation of responsibility for evaluation to experts should be very 
restricted. A small number of specialists in evaluation, preferably developed from ex¬ 
perienced academics, will almost certainly be needed as consultants. I shall not attempt 
here to develop an argument about the nature of duties and the requisite experiences 
and talents of the occasional specialist in evaluation I would like to see developed. 1 
would here simply ask that the preceding paragraph be read as an uncomplicated and 
non-technical approach to the logic of improvement and development. The case for 
dispersal of the major part of the responsibility for evaluation is, however, made below. 

It is easy to explain why evaluation is a key to sound instructional development and 
difficult to propose adequate methods for doing the evaluation. Unfortunately, although 
we are already devoting considerable energies to this problem and we suffer great anx¬ 
ieties lest unfair evaluation systems be introduced, we do not have any very useful con¬ 
ceptual framework within which a sound system is likely to develop. The purpose of this 
section is to suggest a framework which, while incompletely worked out, seems to raise 
the right kinds of questions and may be susceptible to useful development. 

In this account there is quite deliberately no strict line drawn between the evaluation 
of instructional materials, of teaching, and of teachers. There are, of course, instances in 
which a superb teacher is defeated by the circumstances in which he works, or a poor 
teacher can appear to be doing a good job because the system carries him. But it is the 
success of the total system which matters, and if a teacher is a bad lecturer but his stu¬ 
dents achieve a great deal because of the thought and work he has put into alternative 
styles of teaching, then he is a good teacher. In a systematic and holistic approach the 
evaluations must be equally comprehensive. 

One of the most important aspects of a model for an evaluating system is the assign¬ 
ment of responsibility for developing and operating the system. One possibility is the 
current development of specialists in evaluation. However, this trend appears to be with 
us largely because this is the common approach to perceived needs in our society, rather 
than because it has a clear rationale. 

Calling in an established specialist, or developing a new kind to make judgments for 
one, could be called a delegation of responsibility model. However, this is not the only 
choice. In matters of communication between scholars we give only modest powers and 
responsibilities to editors and consider that each scholar must be capable of doing what- 

*In commenting on this paper Mario Creet has said “Comparative evaluation is predicated on the 
assumption that absolute objectives and criteria do not exist. If they did exist, one evaluation would 
be enough. Those who proclaim the freedom of the university in the West might get a jolt if they 
are made to realize that the implicit acceptance of what they do, or the single evaluation of progress, 
are characteristic of orthodoxy no less stringent than Marxism-Leninism, or the Holy Inquisition.” 


42 H. M. Good 


ever is needed in his field, including communicating it. We use here what might be called 
a disperse responsibility model. 

One of the theses of this paper is that while there is a movement toward the delega¬ 
tion model for evaluation, the disperse one is more appropriate. This proposal, and the 
ideas developed from it, are based on a suggestion by Professor L.R.B. Elton of Surrey 
that evaluation of teaching, including of all instructional methods and materials, might 
well work best if done by a system closely comparable to that used for research and 
scholarship. The remainder of this section is devoted to an exploration of this idea. 

The model of evaluation used in research, if less than perfect, has had a great deal of 
success if success be defined not in terms of fixed criteria but rather in adaptive terms 
of providing the corrective feedbacks necessary to keep a process operating with accep¬ 
table and improved effectiveness. It can be described as a peer judgment model. The 
definition of a peer is generally in terms of someone of roughly equivalent background 
in the field who is devoting a high proportion of his or her energy to the subject in 
question. However, because one important feature of the model is its openness — with 
the requirement that evidence for judgments be available for all to see — there.is built 
into the model the potential for a much wider definition of a peer, i.e., anyone who has 
sufficient interest to follow through the argument in detail. 

The peer judgment model of evaluation as used in research has certain preconditions 
for successful operation: 

(1) widely shared assumptions and values which are generally understood within the 

framework of the research in question. 

(2) a substantial population of willing and highly qualified evaluators. 

(3) complete openness of the system with evidence freely available to all. 

Is the research model applicable to teaching? An evaluation system for teaching and 
learning will be more complex than one for research since the research normally focuses 
on a specific problem and may be able to concern itself only with outcomes rather than 
with the way in which outcomes arise. Teaching is an interactive affair in which process 
and outcome are never independent. Moreover, if we attempt to develop a peer judgment 
system in evaluating teaching we shall have to use a very broad definition of a “peer.” 

In matters of scholarship the scholar’s peers are distributed on at least a national, perhaps 
on an international or world wide, basis. In teaching, the peers are chiefly intra-institu- 
tional though they are not strictly so. Inter-university judgments of teaching are widely 
made in relation to preparation of students for graduate study. However the peer is 
probably properly regarded in teaching as coming essentially from the same institution. 

But, the peer need not be defined in terms of the university rank hierarchy or even in 
terms of fellow-teachers. In some situations students will properly be regarded as peers 
and will participate in the evaluation process. In other situations they would not be so 
regarded. This will, at times, be a contentious issue but the debate about how it should 
be resolved should be instructional to those concerned. 

There is nothing in the above which makes the peer judgment model, as used in 
research, fundamentally inapplicable to the teaching process. Indeed it appears to be a 
promising model and one with which the university teachers at least are very familiar. 
Acknowledging that the systems differ in detail and complexity, it seems eminently 
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worthwhile to try to understand the preconditions for success of the research system 
and the applicability of these preconditions to the evaluation of teaching and instruc¬ 
tional programs. Three of the preconditions are discussed below. 

1 — “widely shared and understood assumptions and values” 

This precondition is very largely met in the field of research and also in the field of 
teaching. However, there are exceptions which will, initially at least, be more marked in 
teaching. There is evidence from disciplines with several fundamentally different schools 
of thought that the system will work with only some of the assumptions and values 
shared. There is, however, little hope that it can work without their being widely under¬ 
stood. 

For a clear understanding of assumptions and values it is essential that these be made 
explicit and that staff members feel a responsibility both to make their own objectives 
and values clear and to understand those of their colleagues. 

This precondition is not adequately met now. In both the re-examination of existing 
programs and the design of new ones, much academic debate is little more than a state¬ 
ment of viewpoints. This applies to all levels: course, department, faculty and senate. 

Yet somewhere, if not indeed in several places, it should be possible to provide encourage¬ 
ment (or even gentle coercion) for the debators to apply to their academic discussions 
the same level of critical thought that they would insist on in a research paper. Two 
specific approaches to achieving the objective can be suggested. One involves the insis¬ 
tence by academic administrators that programs be argued critically in, for example, 
curriculum committees. The other involves encouragement of those who have already 
indicated an interest in examination of the assumptions and values of their discipline 
which receive little attention by their colleagues. It is now easier to find discouragement. 
There must also be a well publicized change in the rewards system of the university. All 
of these steps could be achieved in large measure by senior administrative personnel 
asking the right questions or, when affairs are complicated, they might assign consultants 
to committees to see that the right questions are posed and debated critically. Assign¬ 
ment of such consultants would provide an exercise for those inclined to this type of 
analysis of their subjects and a stimulus to their continued study in this difficult but 
essential area. 

2 — “A substantial population of willing and highly qualified evaluators” 

A major factor in the success of the research evaluation system is that virtually every 
academic is willing - indeed eager - to play a part. Evaluation of other persons’ research 
is the very essence of the intellectual game and to be asked to play an important role in 
the evaluation hierarchy as a member of a committee on research, as a journal reviewer or 
thesis examiner, is a much sought after distinction closely related to the rewards system. 
Even for those who do not aspire to promotion within this evaluation hierarchy there is 
a feeling, based on years of indoctrination and training, that willingness to play this role 
is part of one’s duty to the discipline. 

The situation is very different for teaching. There has been (though it is much less 
marked now) the feeling that this is not a primary responsibility, that to attempt to eva- 
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luate someone else’s teaching is an impertinence, that there are no useful or valid criteria, 
and that the whole idea is better forgotten. In contrast to research evaluation there is a 
negative indoctrination, a lack of training, and a lack of incentive or even a disincentive. 

There is also a fairly conspicious lack of expertise. The establishment of criteria for 
good teaching, and the development of methods of evaluation are not quite as unexplored 
fields as most university staff believe. It can, therefore, be argued that there are two parts 
to this precondition: willingness and competence. Both are deficient but it should be 
made clear that any lack of competence is based simply on the fact that the total effort 
put into study of how to evaluate teaching has been to date quite trifling in comparison 
with the effort which has gone into the study of how to evaluate research. 

This two-part precondition is emphatically not met. Yet it is a, if not the, central, 
issue. What action could be taken to increase the extent to which it is met? 

In trying to answer this question it is worth noting that the research evaluation model 
does not involve any “experts.” It depends upon widespread, essentially universal, par¬ 
ticipation. University teachers have, probably correctly, been suspicious of the expert 
in evaluation or in education generally. But they have been unwilling to accept that 
avoidance of “experts” in a difficult field requires a considerable level of expertness 
and effort on the part of most practitioners. 

3 - “Openness of the system” 

Traditionally, university teaching has been far from open and it remains essentially 
a private operation between a staff member and his or her students. This traditional 
pattern is, however, changing fairly rapidly and, in those cases where a course is produced 
by a team working together to produce the whole course (utterly different from a series 
of course fragments given seriatim and independently by several professors), there has 
to be openness about exactly what each proposes to do, why he has chosen that way, its 
implications for all other parts of the course, criteria for success, etc. 

To achieve openness requires a major effort in which there is pressure toward fulfil¬ 
ling all the preconditions here discussed. While very time consuming the team approach 
to course design is a major change in the style of university teaching and one which can 
contribute a great deal to meeting the preconditions for successful evaluation. 

If one is going to apply the research model with regard to openness to teaching, it is 
worth noting that the stress on openness in research is greatest in the evaluation of the 
plan and in the judgment of final success. There is little if any attempt to sample the 
way a scholar is working by looking over his shoulder in the laboratory or library to see 
whether he is doing a good job in the detail of day to day work. In some ways, but not 
all, such minute supervision (which would be regarded as both unnecessary and improper) 
is a parallel to the sampling of a teacher’s performance in one or two lectures. Openness 
of lectures is probably desirable but it gets at only superficial technical aspects of the 
teaching process. 

Therefore two important approaches used extensively in evaluation of research which 
could be more used in teaching are: 

1. Examination and evaluation of the plan for courses including objectives, plan of 

attack, and plans for evaluation. 
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2. Student examination papers reviewed in the light of stated objectives. 

Evaluation of teaching is also more complex than is evaluation of research in that the 
objectives of teaching are more varied. They must cover discipline, institutional and 
student goals. Evaluation to provide a feedback for improvement in a systematic process 
cannot afford to ignore any of these though they may be weighted differently in different 
institutions. 

Teaching — The Choice of Method: Instruction depends upon the two distinguishable, if 
often overlapping, processes of presentation of material and of interaction. Presentation 
is one directional. Interaction is two directional. The totality of the one directional 
system for presenting material is herein referred to as the textual system regardless of 
the nature of the method of delivery. These two parts of the style the textual and inter¬ 
active components will now be discussed. 

The variety of ways in which material can be presented is large — books, printed 
handouts, tapes, slides, T.V., films, computers, lectures. This variety has been increasing 
steadily as a result of technological advances. It brings with it increasing flexibility with 
regard to time, and place, and number — all problems which have been critical to uni¬ 
versities in recent years. It is not, therefore, surprising that the system for presenting 
material, which absorbs an enormous part of universities’ total resources, has been a 
particularly fertile ground for experimental teaching projects, and that technology-flexi¬ 
bility-cost issues, as they relate to textual materials, have provided the principle incentives 
for experimentation. Indeed this relationship is the basis for the popular misconception 
that instructional development is only the use of technologically novel materials. 

Many of the experimental projects supported by universities over the past decade 
have been directly concerned with the lecture which has been included above in the list 
of undirectional or textual materials but must, of course, also be considered potentially 
at least as a part of the interactive system. This dual nature of the lecture is a major 
reason for the arguments about its virtues and shortcomings. Admittedly, there are 
difficulties in separating the textual and interactive aspects of a lecture since in some 
circumstances they are closely interwoven and interact in subtle but important ways. 
However, if one is to choose textual materials according to their advantages for the 
specific job to be done the lecture must be looked at in relation to presentation of ma¬ 
terial and again in relation to interaction. 

Not only do different forms of texts have very different advantages and disadvantages, 
they have also widely different costs. The book, for example, is compact, tightly organiz¬ 
ed, easily indexed, accurate, portable, durable, and cheap if published in sufficient 
numbers. It is, however, lacking in pacing and emphasis, may be too concentrated, is 
limited in the range of visual presentations possible, and has no personality impact. The 
lecture (and let us suppose for the sake of argument that we are considering one which 
is purely textual) has pacing, emphasis and personality impact but is often loosely 
organized, inaccurate as remembered or recorded, ephemeral, inflexible as to place and 
time, difficult to index, and expensive-to-outrageous in cost per unit of instruction. 
However, even if summary statements like this can be accepted, the choice of the medi¬ 
um in which specific textual materials can best be provided is a complex one and needs 
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to be looked at very critically; the more so since different forms are appropriate for 
different materials and also for different students. 

There is, at present, a trend away from lecturing and there are many experimental 
programs, and some established ones, using the “new media” of film, television, tape- 
slide, etc. to replace lectures. This is a sector of instructional development which is very 
active and there have been both successes and failures. It is a legitimate area, although 
in many cases programs have been based on a superficial analysis of the range of possi¬ 
bilities and relative advantages of each method and have been aimed at a narrow and 
incomplete set of objectives. Quite predictably many were found wanting when looked 
at comprehensively and have been discarded; the baby being routinely thrown out with 
the bath water. The long term savings and the increased successes of the iterative develop¬ 
mental approach would, over the past ten years, have been enormous. 

Because different kinds of textual materials have different advantages, because each 
instructional program has a complex set of objectives, because classes are heterogeneous, 
it follows that most programs can profitably use a mix of materials and/or offer students 
alternative modes of access to the textual materials. The mix will be based on the merits 
of methods for specific jobs and/or cost constraints. Trade-offs aimed at optimizing the 
effectiveness of the total package will, therefore, be necessary. 

The choice of an appropriate mode of presentation — book, lecture, etc. — is not, 
however, the end of the textual issue, which is a vital part of the overall methods issue. 
How does one get the best materials in whatever modes are chosen and how does one 
get them published? 

Implicit, but rarely expressed, in efforts to improve the textual package is a criticism 
of the current system. To some extent this is based on scholars writing for colleagues 
instead of for students and to some extent to an almost complete dependence on com¬ 
mercial publishing which understandably is constrained by the need to turn a profit. 

But if the text-producing industry is doing a less than ideal job and if this is reducing the 
effectiveness of the educational process significantly then we need to consider whether 
we might not gain greatly from an integrated instructional program with texts of all 
kinds eligible for subsidy from general educational funds. The only case in which this 
has been done systematically is Britain’s Open University — an institution which is 
having an enormous impact in areas which may appear quite separate from its role as 
an “open” (i.e., no prerequisites) institution. 

The Open University was designed to serve students at a distance whose principal 
personal contact with staff would be with tutors rather than with the course designers 
and the writers of textual materials. The need to separate out clearly the textual and 
interactive phases of teaching was, therefore, especially marked. In addition, the question 
of how to choose the best blend of textual materials, how to get the best possible 
quality produced, and how to publish this material all had to be faced. 

The approach used by the Open University involved: 

(a) adoption of the principles of systematic course development. 

(b) creation of a course team for each project, the team to include subject experts, 

educational technologists, editors, media experts, etc. 

(c) use of a mix of modes of presentation — books, T.V., lectures, tapes, etc. — 

chosen after careful examination of costs and advantages. 
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(d) publication of much of the material by the Open University itself. (It has become 
the largest educational publisher in Britain.) Commercially available materials are, 
of course, used when suitable. 

Thus, the Open University, in its effort to think the educational process through 
systematically, found itself deviating very greatly from traditional patterns in the im¬ 
portant areas of what textual materials to use, how to get the best possible ones prepared, 
and how to publish them. The high quality of their textual materials has been generally 
recognized. They have in fact made the case for the application of the systematic me¬ 
thod to the textual component of university teaching. But there has, as yet, been little 
recognition of the wide range of implications of their success. 

The range of approaches to interactive instruction is much more limited than that 
for presentation of material. Interactive instruction implies an essentially one-to-one 
responsive situation though this can be achieved in a group if it be small enough to 
provide a series of one-to-one encounters. The interaction is perhaps greatest between 
teacher and student but the student-student relations can be extremely productive. 

There is considerable literature on small group and interview techniques and it is 
clear that the value of time spent in interactive activities can be increased greatly by 
systematic planning of what specific components of the total instructional program are 
best done in this way and how to get the best results with given resources, especially of 
time. Instructional development can therefore contribute to both the effectiveness and 
efficiency of the interactive component. 

Equally it can contribute to the organization of the over-all program so that this 
vital component of instruction is not short-changed as it sometimes is by invoking un¬ 
necessary constraints. Present resources could quite easily provide for more interactive 
instruction if this were considered a high priority. In an earlier section on the advantages 
of Instructional Development, a chemistry course revision was cited in which a redevelop¬ 
ment of the textual materials could permit the redeployment of some half dozen senior 
staff into interactive instead of presentational activities. This example merits careful study. 
It focusses on the possibility that the small-to-medium sized lecture section is rationalized 
on the basis of its having an interactive component which is lacking in the more cost- 
effective large lecture system. But the rationale only holds if the interactive teaching de¬ 
livered this way is more effective than it would be if the textual and interactive teaching 
were separate clearly as in the Open University, or the cited Chemistry Program. This 
comparison is rarely attempted. 

There is at the moment a great deal of talk about the need to do more re-deployment 
of this type. But so long as there is no willingness to look at possible savings in the effort 
put into purely presentational or textual instruction this will be little more than empty 
rhetoric. University teachers are busy now. They cannot simply be loaded with more 
work. The answer can only come from a reappraisal of where the time resource can do 
most good. 

IV. Conclusions and Recommendations 

It would be unfair to suggest that Canadian universities never go through careful and 
systematic development of programs. But only from a position of profound ignorance 
can it be argued that a substantial proportion of day to day decisions regarding their 
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academic management is made according to any tightly reasoned plan. Because vested 
interests and politics are so prominent, decisions have their roots in history and in chance 
concatenation of forces. But chance will not do. We need choice — and careful choice at 
that. 

Careful choice, after review of alternatives, is the essence of management. Universities 
today demand the right to be self determining. This right carries with it the responsibility 
to be both honest and precise about what the institution is trying to do and how it is 
going about it — a credible match between rhetoric and practice. Much harm is now 
being done to the universities because this match seems less than credible to both students 
and public. The mismatch arises from a failure to use a logical and systematic approach 
in the matters of staffing, rewards system, and instructional programs. The failures of 
the rewards system and the instructional program will be discussed briefly in relation to 
remedial action. The staffing problem was dealt with at some length in Section II and 
will be discussed only as it inter-relates with the rewards system. 

The rewards system provides the most direct, continuous and powerful influence 
shaping the attitudes and priorities of staff. These attitudes are, in sum, essentially those 
of the university. A rewards system without critical evaluation is a farce, and rewards 
not related to institutional objectives are dishonest. 

There is no logical way of sidestepping the evaluation issue - difficult as it may be. 

It is an essential part of the systematic approach to instruction at the lesson, unit, or 
course level. It is an essential part of institutional administration through the rewards 
system. 

There cannot be institutional accountability without critical attention to evaluation 
and rewards, and evaluation implies consideration of effectiveness of the use of teachers’ 
efforts. It is not unreasonable to claim that this aspect of institutional accountability 
deserves as much attention as does financial accountability. In this connection it is in¬ 
teresting to analyze university management systems in terms of accountability for the 
dollar and accountability for the hour. Even Deans and Vice Principals are concerned 
primarily with dollar accountability and the management of, and accountability for, 
the academic process is largely ignored. Nor is this situation likely to change until the 
objectives-rewards systems are matched in a highly visible way. 

As evaluation is improved and applied to the rewards system there must be consistent 
and courageous use of the results. However, the development process cannot be rushed. 
Fear of inappropriate evaluation and a deep mistrust of existing systems is currently a 
more important bar to teachers undertaking systematic course development than is 
disinterest or ignorance of textual resources, or lack of understanding of interactions of 
the components of the system. 

The questions of the specific form of rewards or penalties (salary adjustments, rank, 
leave, teaching loads) which now concern many in the universities are details which have 
to be faced, but they can be left — or handled as at present in an ad hoc way — until 
the evaluation problems are, if not solved, at least incorporated into the system in a 
way likely to evolve a more satisfactory working basis. 

This working basis clearly has to involve wide participation by the university com¬ 
munity if the disperse responsibility model implicit in the democratic management of 
the university is to be honored. 
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What is needed is an approach which is pervasive at all levels, i.e., encouragement of 
analytical questions at departmental level, a better staff evaluation process perhaps 
modelled on that used in research, a system of critical review of departmental briefs at 
Faculty level with firm rejection when the rationale is sloppy, and so on throughout the 
university. 

This “all levels” approach will not work without an initial cadre of people who have 
an interest in the work and a willingness to devote a part of their study time to the 
process of systematic development. 

It seems probable, therefore, that in spite of the arguments for a disperse responsi¬ 
bility (with its implication for a moderate level of disperse competence and interest) 
there will have to be a seeding operation in which some university staff are in effect 
seconded to the duties suggested. This will require resources but is not a revolutionary 
proposal. If a modern university found itself without accountants it would not hesitate 
to hire them and if none were available it would send staff out for a crash program to 
qualify them for the job. No such action is visible with regard to the accounting for the 
deployment of educational resources. 

In summary, the recommended actions are: 

1. Development of a cadre of part time “consultants” to work in departmental or 
faculty contexts as critics of programs and proposals. It is essential that these be 
part-time persons who continue to be directly involved in teaching and desirable that 
they be established and respected members of staff. This proposal could so easily 
lead to the development of a new brand of academic administration based on “spe¬ 
cialists” that care should be taken at the outset to design a program which fosters the 
disperse responsibility model. 

2. Modification of departmental and faculty administration to focus more sharply on 
instructional issues and strategies, probably through a revised evaluation and rewards 
system. 

Both of these steps require strong leadership at the higher levels of the university 
but they do not attempt to work simply through upper administrative channels. All 
levels are involved as they must be if the universities are to justify the wide use of the 
democratic decision making which they espouse so warmly. 

The importance of the evaluation process has been stressed above almost to the 
exclusion of the instructional process. It is true that if the systematic approach is to 
be followed fully then consideration of any problem leads into consideration of the 
whole system. Thus it can be argued that if staffing has been carried out with careful 
regard to institutional objectives and a good evaluation and rewards system operates 
the instructional process will be looked after. However, from a practical viewpoint, the 
reverse route is more likely to be profitable. The support of individual projects to de¬ 
velop instructional materials carefully is probably the most effective way of interesting 
staff in a more critical approach to teaching. An increase in interest in academic manage¬ 
ment as a challenging problem should create an interest in the research model for evalu¬ 
ation, and in a clearer enunciation of objectives. This might influence staff towards a 
distribution in the responsibility triangle more justifiable in terms of institutional 
objectives. 
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Instructional development in the narrow sense of production of specific material is 
therefore a useful first step in the overall process — perhaps the most useful one. But it 
is important to reiterate that haphazard experimentation and random data collection 
are no more effective in development of instruction than they are in the acquisition of 
knowledge. Most funds now allocated to “innovation” are wasted. They are given in a 
pious gesture by university officials who lack either an adequate measure of the problen 
or the courage to act on the measures which they do have. 

To paraphrase Robert Diamond, Assistant Vice Channcellor of Syracuse: the wide¬ 
spread encouragement of small projects (as opposed to the careful selection of a few 
major ones) is an excellent method of advertising administrative commitment to instruc 
tional improvement but many small projects will be of low priority and the results are 
usually unsatisfactory. As argued earlier in relation to the Open University, and to the 
more successful projects in all jurisdictions, a thorough systematic approach works well, 
but half-way measures are costly and likely to be unproductive. 

The general conclusion is that there are huge gaps in the academic management sys¬ 
tem and that simple accountability for use of the major university resources — staff and 
student time — requires more critical accounting than the university now gives or is 
capable of giving on short notice. What is needed is strong leadership toward develop¬ 
ments which will progressively improve the university’s ability to account for its use of 
these resources. 

In providing leadership there must be recognition that instructional development is 
a continuing iterative process. As such it must have institutional support to assure it 
of continuity. Without continuity it remains largely haphazard innovation - possibly 
successful but probably wasteful. In no area is the old adage more applicable “If a thing 
is worth doing it is worth doing well.” 
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